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SUMMARY 



High rates of attrition aMong students In Undergraduate Navigator Training (UNI) continue to 
be a aajor concern for Air Training CoHund, In an effort to address this problem, research was 
conducted by the Air Force HuMan Resources Laboratory to reduce attrition by Improving the laethod 
by which candidates are selected. Currently, UNT students are chosen primarily on the basis of 
their scores on the Navigator-Technical (Nav*Tech) composite of the Air Force Officer Qualifying 
Test (AFOQT). T^e present effort sought to determine the extent to which scores on an 
experimental selection battery, known as the Basic Navigator Battery (BNB), added to the quality 
of selection screening provided solely by the Nav-Tech composite score. 

Three selection models were evaluated against four UNT performance standards. The three 
models examined consisted of AFOQT Nav-Tech composite score only (Model 1), AFOQT Nav-Tech 
cofiposlte score plus other AFOQT composite scores (Model U), and AFOQT Nav-Tech composite score, 
other AFOQT composite scores and BNB subtest scores (Model III). The four measures of UNT 
performance were training outcome (graduation/elimination), average classroom lesson score, 
average simulator lesson score, and average flying lesson score. 

Model III was found to have the highest predictive accurac;y for all four performance 
standards. Compared to Models I and II» Model III Identified more unqualified candidates. This 
model was also particularly good In predicting classroom and slmul^t r performance, suggesting It 

be useful as a diagnostic aid to Identify students who may need remedial help during training. 

The results of this effort provided preliminary evidence of the usefulness of the BNB In 

reducing UNT attrition through Improved selection. Further research Is necessary, however, to 

obtain better estimates of the stab111t;y and validity of this test and to develop new forms of 
the BNB before It could be used operationally. 
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BASIC NAVIGATOR BATTERY: AN EXPERIMENTAL SELECTION 
COMPOSITE FOR UNDERGRADUATE NAVIGATOR TRAINING 



I. INTRODUCTION 

Unacceptably high rates of attrition anong students In the Undergraduate Navigator Training 
(UNT) Prograa have been a continuing concern for the Air Training Comnd (ATC). Loss of 
studenU In UNT represents a substantial cost to the Air Force both In terns of the waste of 
training resources and the subsequent difficulty In neeting operational Manpower needs. While 
retention of all entering trainees Is virtually 1in)oss1ble due to a variety of factors beyond the 
control of the Air Force, lower rates of attrition ma^ be achieved by Improving the 
screening/selection of trainees, and designing currlculuM and Instructional aethods to optimize 
student learning. ThU report focuses on the for«er of these tw approaches to alnlalzlng the 
loss of students In UNT. 

The Bost obvious ■et^od for reducing attrition anong UNT trainees Is through the Initial 
selection of highly qualified candidates. Since World War II, trainees have been selected 
priMrlly on the basis of their scores on the Navigator-Technical (Nav-Tech) composite of the Air 
¥orct Officer Qualifying Test (AFOQT). The validity of this paper-and-pencll test for predicting 
success (I.e., passing) In UNT has been supported by previous research (Miller, 1966; Valentine A 
Creager, 1961). 

In spite of the significant predictive validity of the AFOQT, continuously high trainee 
attrition rates have provided the liapetus for research to Identify supplenental and/or 
alternative selection InstruMnts. Based on their nultl-faceted Investigation of the causes of 
turnover In UNT, DeVrles, Yaklw), Curtin, and McKenzle (1975) recoMiended the iMpleaentatlon of a 
selection systea which utilized AFOQT scores, as well as personality. Interest, and cognitive 
ability measures. Subsequently, Valentine (1977) conducted a study designed to reassess the 
predictive validity of the AFOQT following changes In the UNT currlculuM, and also to exaalne the 
potential validity of an experimental selection battery consisting of personalltyt Interest, 
biographical, attitude, and cognitive ability measures. iMproveMnts In the predictive validity 
of the AFOQT were achieved with so«e of the cognitive ability Masures. but not with any of the 
noncognltlve measures. On the basis of this research, Valentine reconended a revision of the 
Nav-Tech composite of the AFOQT to Incorporate those measures which were significantly related to 
success In UNT« 

The purpose of the present effort was to examine the potential usefulness of an experimental 
navigator selection battery* the Basic Navigator Battery (BNB). In Increasing the predictive 
validity of the current navigator selection system. Specifically, this effort attempted to 
determine which. If ai\y, of the subtesU of the BNB added significantly to the overall predictive 
validity of the Nav-Tech composite of the AFOQT. The present study represented a significant 
departure from previous Investigations of navigator selection In that. In addition to the 
standard pass/fall criterion of success In training, lesson grades for each phase of UNT were 
also used as criterion measures. Use of these lesson grades provides a more detailed analysis of 
the relationships between the predictors of Interest (I.e., AFOQT, BNB) and specific phases of 
UNT training. 



II. APPROACH 

Subjects 

Air Force officers entering UNT (course N-V6A-F) at Mather AFB, California, during the 1980 
calendar year (CY80) served as subjects. This sample was comprised of UNT classes 80-18 through 
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81-17, Of the 800 officers who went through the progran during this period, relatively complete 
data were available for 544, who comprised the saaple analyzed in this report. 



Predictor Vtrtables 

Twu sets of predictor variables were exanined. The first consisted of the five coivosites 
(Pilot, Navigator-Technical, Academic Aptitude, Verbal, and Quantitative) of the AFOQT. Connon 
■etHc percentile scores (Roach « Rogers, 1982) for these subtests were obtained from historical 
personnel records. The second set of predictors was the percentage scores from the five subtests 
of the BN8. The BNB was aMnistered to all subjects 1 week prior to the beginning of their 
assigned training class. The BNB subtests were as follows: 

Prg-Nav (PE7935) — a 40-ite«, aultiple-choice test neasuring nathenatical reasoning 
ability within the context of navigational problen solving. This test was originally 
designed as a diagnostic instrunent for identifying stbdents in need of remedial 
Mthenatics training. 

Infonmtion Processing (PE7935) — a 30-iten test of perceptual reasoning, divided 
equally between hidden figures items and figure analogy items. Hidden figures Is a 
template-matching task that requires the subject to locate a sicple geometric figure 
embedded within a complex drawing. Figure analogies is a test of syc^bolic logic in 
which the subject must first discern the logic of a relationship (i.e., shape, size, 
axis rotation) between a pair of sample geometric figures, and then apply this logic in 
attempting to match a target stimulus figure with one of five alternatives. 

Obstacles and Remedies (PE7933/7934) - an 80-item speeded test that measures the 
subject's ability to follow a set of proc iures. The subject's task is to apply a set 
of rules and priorities in removing sets of obstacles encountered along an imaginary 
path. 

Simulated Navigation Mission (PE7936/7938) - a test that renJiires the subject to plot 
the location of an aircraft at 10 points during a "mission." Radio signals generated at 
two locations, flying time, wind speed, wind direction, and altitude are provided to the 
subject for use in calculating aircraft locations. At each location, the subject must 
compute both the distance travelled since the last reading and the aircraft spe^^l, as 
well as answer questions about fiiel consuq)tion. 

Rotated Letters (PE7932) — a speeded test in which the subject must Judge whether each 
of 288 pairs of rotated letter: is in the same orientation or whether they are mirror 
images. 

Descriptive statistics for all of the predictor variables are presented in Table 1. 



Criterion Variables 

Two types of variables were used as indicators of UNT perfonnance. The first was a single 
dichotomous measure of overall training outcome, scored 0 for elimination, 1 for graduation. The 
second type of indicator consisted of summary variables for each of the three phases of training 
(i.e., classroom, simulator, and flying lessons). These sunnary variables were formed by 
averaging the individual lesson grades for each phase of training. Descriptive statistics for 
these summary scores and the individual lesson grades are provided in Table 2. 



2 



10 



Table 1 , Descriptive Statistics for Predictor Variables 





ncin 




II 

N 


AFOQT Co^iosltes 








1. Pilot 


61.33 


22. 75 


544 


2* Navigator-Technical 


56.07 


22. 55 


544 


0, AcaoeKic ^tituoe 


64.17 


22.71 


544 


4* verDai 


63.00 


23.84 


544 


d« i^antivavive 


62.45 


22.52 


543 


Basic Navigator Battery Coqiosltes 








1. Pre-Nav 


69.34 


15.03 


544 


2. InfoHMtlon Processing 


54.72 


17.29 


544 


3. Obstacles and Remedies 


58.38 


18.51 


543 


4. SiMilated Navigation Mission 


50.86 


22.10 


544 


5. Rotated Letters 


68.93 


15.67 


542 


Table 2. Descriptive Statistics for CHteHon VaHables 




(Individual Lesson firades and Co^ioslte Lesson Grades) 






Mean 


SD 


M 


UNT Lamaiic 








1. Aerospace Physiology 


92.57 


5.58 


544 


2. AlHNnshlp 


93.34 


6.18 




3. Advanced Almanshlp 


89.31 


7.50 


541 


4. keather 


93.16 


7.09 


540 


5. Navigation Procedures 


88.70 


11.07 


538 


6. Basic Navigation 


93.03 


6.27 


509 


/• Day Celestial 


85.08 


11.64 


dfi7 


8. Night Celestial 


92.18 


6.80 


469 


UNT SiMilator Lessons 








9. Navigation Procedures T-45 


86.81 


13.95 


527 


10. Basic Navigation T-45 


81.05 


12.68 


505 


11. Tactical Navigation T-45 


88.31 


9.41 




UNT Flying Lessons 








12. Radar T-43 


85.42 


10.52 




13. Day Celestial T-43 


84.59 


10.75 


475 


14. Night Celestial T-43 


86.39 


10.39 


467 


15. Tactical Navigation T-37 


89.43 


8.65 


460 


UNT Graduatlon/E11«1nat1on* 


.84 


.37 


544 


UNT Coaposlte Scores 








1. Classroon Lessons 


90. 77 


5.14 


538 


2. Simulator Lessons 


85.29 


8.52 


505 


3. Flying Lessons 


86.62 


5.67 


467 



'Coded 1 If graduate; 0 If ellMlnee. 
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Daf Analysis 



The dat« were analyzed with hierarchical multiple regression procedures. A separate analysis 
was conducted for each of the four neasures of UNI performance (I.e., overall training outcome, 
classroom grades, simulator grades, flying grades). Entry order of the predictor variables In 
each regression was determined as follows. The Nav-Tech coiposlte of the AFOQT was entered on 
the first step of each regression, since It was used to select candidates for UNT. The four 
remaining AFOQT composites were then analyzed to determine If they Individually added to the 
predictive validity of the Nav-Tech coit>os1te. The significance of the contribution of each of 
the four AFOQT composites to the Nav-Tech-based prediction equation was assessed with F-tests. 
Those composites which added significantly to the amount of criterion variance accounted for were 
subsequently entered Into an AFOQT^-based prediction equation In descending order of the magnitude 
of their contribution. 

After delineating the final AFOQT-based prediction equation, the five subtests of the BNB 
were assessed for their potential contributions toward Increasing the auiount of variance 
accounted for In each of the UNT performance measures. As In the preceding analysis, F-tests 
were used to determine the significance of the Increase In predictive validity resulting from the 
addition of each BNB subtest to the AFOQT equation. Those subtests which produced a significant 
Increase were entered Into a final prediction equation, again In descending order of the Increase 
produced. 

It should be noted that the correlations used In the regression analyses were not corrected 
for the restriction In test score variance which Is likely to result from the use of a sample 
that Is pre-selected on one of the predictors of Interest (I.e., AFOQT Nav-Tech scores In the 
present stu4/). Since this stu(ty used a design In which all subjects had been pre-selected for 
UNT prior to testing on the BNB, It was to be expected that the magnitude of the validity 
coefficients would be attenuated to a certain degree. Thus, It Is reasonable to assuie that In a 
sample unrestricted by pre-selectlon effects, the strength of the resulting validity coefficients 
would be larger than those evidenced In the present stucly. However, the procedures used In this 
stu4y do address the primary concern; I.e., whether or not the BNB has the potential to Improve 
the selection of UNT candidates over and above that which can be expected with the AFOQT already 
In use. 



III. RESULTS AW) DISCUSSION 

The zero-order correlations among all predictor and criterion variables are presented In 
Table 3. All of the AFOQT and BNB subtests, with the exception of the BNB Rotated Letters 
subtest, were found to correlate significantly (p < .05) with the graduation/elimination 
(grad/ellm) performance measure. Similar results were obtained for the classroom lesson grades 
criterion. All AFOQT composites and Mh subtests correlated significantly (p < .05) with 
simulator lesson grades, and all but the AFOQT Verbal coiposlte correlated significantly with the 
flying grades criterion. 

The results of viie hierarchical regression analyses are presented In Tables 4 through 7. 
Overall, these tables provide evidence that the validity of the AFOQT Nav-Tech composite can be 
1^>roved significantly by Including scores from selected AFOQT coiposltes and BNB subtests In the 
selection decision. 

Table 4 sumrlzes the results for the prediction of the overall training outcome. This 
table shows that adding three predictors (AFOQT Quantitative, BNB Pre-Nav, BNB Obstacles and 
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irtble_3. Zero-Order Correlations Among Predictor and Criterion Variables 
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6. Pre-Nav 


47 


53 


53 


36 


56 


- 












7. Information 
























Processing 


37 


46 


31 


27 


31 


43 


- 










8. Obstacles and 
























Remedies 


24 


35 


18 


10 


26 


30 


19 


_ 








9. Simulated Navi- 
























gation Mission 


39 


51 


44 


29 


50 


53 


33 


34 








10. Rotated Letters 


23 


32 


10 


08 


13 


23 


26 


31 


20 






11. Grad/Elim 


16 


17 


16 


10 


19 


25 


14 


17 


21 


02x 


- 


12. UNT Classroom 
























Lessons Grade 


30 


37 


36 


22 


40 


49 


25 


23 


37 


Oix 


48 


13. UNT Simulator 
























Lessons Grade 


26 


29 


24 


11 


32 


42 


21 


26 


33 


14 




14. UNT Flying 






















Lessons Grade 


21 


24 


09* 


01 « 


14 


27 


18 


'7 


22 


09* 


20 



12 13 



51 
33 



42 



14 



Note. Decimal points ooiitted. For each correlation, n varied from 536-547. All correlations 
significant for£. < .01 unless otherwise noted. 
^Nonsignificant correlation. 
*£ < .05. 



Table 4. Prediction of Graduation/Elimination from UNT 

R r2 



Predictors 



Change 



AFOQT Nav-Tech 

AFOQT Nav-Tech + 
AFOQT Quantitative 

AFOQT Nav-Tech + 
AFOQT Quantitative + 
BNB Pre-Nav 

AFOQT Nav-Tech + 
AFOQT Quantitative ^ 
BNB Pre-Nav ^ 

BNB Obstacles and Remedies 



.15 



.023 



.20 .039 



.25 .063 



.27 .071 



.016' 



.024** 



.007* 



Note. N^ ■ 542. 
*££ .05. 
**p < .01. 
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Remedies) to the AFOQT Nav-Tech-based prediction equation significantly raised the coefficient of 
multiple correlation (K) from .15 to .27, thus Increasing the amount of variance accounted for 
(£2) In the graduation/elimination outcome measure. 

Table 5 sumnarlzes the results of the analysis for the prediction of the UNT classroom lesson 
grades performance. As In the analysis of the grad/ellm outcome, using the AFOQT Quantitative 
convosite and both the Pre-Nav and Obstacles and Remedies subtests of the BNB significantly 
Increased the predictive validity provided solely by the Nav-Tech cooi)osite. The BNB Rotated 
Letters subtest also provided a significant increase in overall predictive validity. The 
Nav-Tech composite alone had a correlation of .36 with classroom performance, whereas a 
prediction equation incorporating the Nav-Tech composite plus the other four predictors 
identified by this analysis had a multiple correlation of .54. 

The results of the regression analysis for the prediction of the simulator lesson grades are 
presented in Table 6. Two AF QT composites {Qua.ititative and Pilot) and three BNB subtests 
(Pre-Nav. Obstacles and Remedies, Simulated Navigation Mission) produced significant increases to 
the level of prediction provided solely by the Nav-Tech composite. Addition of the two other 
AFOQT coiH)Osites significantly raised the correlation from .29 to .35. Adding th. three BNB 
subtest scores to the prediction equation further significantly raised the correlation to .46. 

Table 7 presents the results of the regression analysis for the prediction of flying lesson 
grades. The Nav-Tech composite had a correlation of .24 with these grades. The addition of the 
AFOQT Verbal cofi^)Osite and the BNB Pre-Nav subtest significantly raised the correlation to .33. 



Table 5. Prediction of Average UNT Classroom Lesson Grades 



Predictors 



R^ Change 



AFOQT Nav-Tech 



.36 



.127 



AFOQT Nav-Tech + 
AFOQT Quantitative 



.43 



.182 



.055** 



AFOQT Nav-Tech + 
AFOQT Quantitative ^ 
BNB Pre-Nav 



.51 



.265 



.083** 



AFOQT Nav-Tech + 
AFOQT Quantitative ^ 
BNB Pre-Nav + 
B.JB Rotated Letters 



.53 



.283 



.018** 



AFOQT Nav-Tech ^ 

AFOQT Quantitative ^ 

BNB Pre-Nav ^ 

BNB Rotated Letters ^ 

BNB Obstacles and Remedies 



.54 



.289 



.006* 



Note. N^ « 534. 
*p < .05. 
**p < .01 . 
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Table S. Prediction of Average UNT Slwjiator Lesson Grades 

R r2 ' 



Predictors 



Change 



AFOpT Nav-Tech 

AFOQT Nav-Tech 
AFOQT Quantitative 

AFOQT Nav-Tech + 
AFOQT Quantitative ^ 
AFOQT Pilot 

AFOQT Nav-Tech ^ 
AFOQT Quantitative ^ 
AFOQT Pilot ♦ 
BNB Pre-Nav 

AFOQT Nav-Tech + 

AFOQT Quantitative + 

AFOQT Pilot + 

BNB Pre-Nav ^ 

BNB Obstacles and Remedies 

AFOQT Nav-Tech + 
AFOQT Quantitative ^ 
AFOQT Pilot + 
BNB Pre-Nav ^ 

BNB Obstacles and Remedies ^ 
BNB Simulated Navigation Mission 



.29 



.082 



.34 .116 



.35 .126 



.43 .187 



.45 .205 



.46 .214 



.034* 



.009* 



.061** 



.018* 



.009* 



Note. H - 505. 
*£ < .05. 
**p < .01. 



Table 7. Prediction of Average UNT Flying Lesson Grades 



Predictors 


R 


r2 


Change 


AFOQT Mav-Tech 


.24 


.058 




AFOQT Mav-Tech + 
AFOQT Verbal 


.26 


.068 


.01* 


AFOQT Mav-Tech + 
AFOQT Verbal * 
BMB Pre-Mav 


.33 


.108 


.04** 



Note. M^ ■ 467. 
*£ < .05. 
**£ < .01. 
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IV. CONCLUSIONS AND RECOmENOATIONS 
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The major fint'ings of this effort can be s'jnmarlzed as follows: 

1. The Nav-Tech composite of the AFOQT was a significant predictor for all four UNT 
performance criteria examined. 

2. The predictive validity of the Nav-Tech composite was significantly increased for each 
cHteHon measure by the addition of at least one other AFOQT composite. The Quantitative 
composite provided a significant increase in the predictive validity for three of four criteria 
(i.e., grad/elim, classroom grades, simulator grades). 

3. The predictive validity of the Nav-Tech and the other selected AFOQT composites was 
significantly increased for all criteria by the addition of certain BNB subtests to the 
prediction equation. The Pre-Nav subtest produced a significant increase in predictive validity 
for all four criteria, while the Obstacles and Remedies subtest significantly incremented the 
predictive validity for three of four criteria (i.e., grad/elim, classroom grades, simulator 
grades). 

These results provide preliminary evidence that the predictive validity of the current selection 
system for UNT trainees can be improved by adding information from the AFOQT Quantitative 
comiosite and at least two BNB subtests to the selection decision process. The practical 
iiplications of the potential increase in predictive validity are illustrated in Table 8. This 
table shows the percentage of trainees in the present sample which would have been correctly or 
incorrectly rejected using the three prediction models identified in the analysis of UNT 
graduation/elimination. The upper portion of the table sumnaHzes the percentage of eliminees in 
the saivle who wuld have been correctly rejected (i.e., true negatives) by rank-ordering the 
candidates by their final prediction equation scores and then using one of three different 
percentile scores as a cutoff point. These percentages, therefore, are indicative of the 
relative efficiency of each model at correctly screening out unqualified UNT candidates. Model 
I containing only the AFOQT Nav-Tech composite, screened the lowest percentage of eliminees at 
all percentile score levels. Model III, containing the AFOQT Nav-Tech and Quantitative 
composites, and BNB Pre-Nav ano Obstacles and Remedies scores, correctly screened the largest 
percentage of unqualified candidates at all the percentile scores. 

The lower portion of Table 8 provides data on the percentage of UNT graduates who would have 
been incorrectly rejected (i.e., false positives) by each model for three percentile cutoff 
scores. Because these percentages represent each model's efficiency in minimizing the incorrect 
rejection of qualified candidates, smaller percentages indicate greater efficiency. These 
comparisons revealed very little difference among the three models at aiv of the percentile 
cutoff scores. Essentially, all three models were equivalent in their rates of incorrectly 
rejecting candidates who would successfully complete UNT. Overall, considering the correct and 
incorrect rejection characteristics of all three models. Model III appears to be the best 
available selection system at the present time. 

Although these results are encouraging, incorporation of the changes suggested by this study 
into the present system for selecting navigator trainees would be premature at this time. 
Additional research on the BNB is needed in two areas. First, the psychometric characteristics 
of the BNB subtests, such as internal consistency, should be assessed at a more in-depth level 
than was possible in the present study since only total test scores, not individual item data, 
were available for each subtest. Second, the unique predictive validity of the BNB subtests 
should be assessed when they are given at the same time as the AFOQT. In the present study, 
because of testing limitations, the BNB was given te»UNT candidates 1 week prior to their 
training whereas the AFOQT wai administered, for operational selection purposes, several months 



16 



Table 8 , Ccwparative Rejection Rates for Three UNT Selection Nodels 



Percentile 

10th 20th 3Qth 

L Percentage of Actual Himinees Rejected 

Model L AFOQT Nav-Tech 

Model II. AFOQT Nav-Tech 

AFOQT Quantitative 

Model III. AFOQT Nav-Tech 

AFOQT Quantitative ^ 
BHB Pre-Nav ^ 

BNB Obstacles and Remedies 27. U 43.5% 55.3% 
II. Percentage of Actual Graduates Rejected 

Model I. AFOQT Nav-Tech 7.4% 16.4% 25.6% 

Model II. AFOQT Nav-Tech ^ 

AFOQT Quantitative 7.7% 16.8% 25.6% 

Model III, AFOQT Nav-Tech ^ 

AFOQT Quantitative + 
BNB Pre-Nav ^ 

BNB Obstacles and Remedies 7.7% 16.8% 24.1% 



11.5% 28.7% 41.4% 

16.3% 39.5% 44.2% 



prior to training. The effects of this tine difference could not be accounted for in the present 
effort. A follow-on stu4y with concurrent adninistration of the AFOQT and Btffi could provide the 
data necessary to address both areas of concern. 



Three recoMiendations for future navigator selection research are offered. First, a stu(|y 
should be conducted in which simultaneous data collection on all predictor measures is possible. 
Optimallyp all tests would be given to UNT applicants prior to their operational selection. This 
type of design would allow for an assessment of the unique predictive validity of the BNB and 
also provide an estimate of the distribution of BNB test scores within the applicant pool. 
However, because of applicant testing time constraints, this approach might not be feasible. A 
fall-back approach would be to re-test UNT selectees on the AFOQT and administer the BNB Just 
prior to training. Although this design would not provide a good estimate of the applicant pool 
BNB score distribution, it would answer the major question about the unique pr^^dictive validity 
of the BNB when administered with the AFOQT. 

The second recommendation is to begin the development of new forms of the BNB. The current 
version could not be used operationally because it has been available for several years as an 
uncontrolled test. Therefore, it is likely that an unacceptable level of test compromise (i.e., 
applicants having access to test questions) would prevent its operational use. For operational 
use, new versions would have to be developed and protected from compromise such that applicants' 
scores would reflect their true abilities rather than their fam^Marity with the test. 

The third recommendation is more general. Future efforts in navigator selection research 
might benefit from ongo'n^ research in the field of computerized selection testing. As part of a 
program to ii|)rove pilot selection, a computerized test $ystem has been developed to measure 
informtion processing capabilities, psychomotor abilities, and certain personality 
characteristics. A similar computerized system, adapted specifically for navigator selection, 
might further increase the accuracy of prediction over that obtainable from conventional 
paper«and-pencil tests such as the AFOQT and BNB, 
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